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Key Findings 



• Limited capacity to monitor. Many states (38) are unable to monitor “to a great extent” 
the quality and effectiveness of SES providers; only 1 0 states reported being able to do 
so. The greatest capacity challenges for states in meeting this federal SES monitoring 
requirement are insufficient numbers of staff and inadequate federal funding. 

• Use of criteria in law. Almost all (between 47 and 49) of the state education agencies 
we surveyed reported using the criteria required by NCLB law and federal guidance to 
review and approve applications from potential supplemental service providers. These 
criteria are intended to ensure that providers are financially sound, have a record of effec- 
tiveness, use research-based strategies, provide services consistent with district instruc- 
tion, and adhere to health, safety, and civil rights laws. 

• Frequent updating, NCLB requires states to promote maximum participation of SES 
providers so that parents have as many choices as possible. Therefore, it is important for 
states to provide parents and school districts with a current and accurate list of SES 
providers that they can choose from. On our survey, 20 states said they review new SES 
provider applications more often than once a year (the minimum required by the NCLB 
law), and 22 states reported updating their SES provider lists more than once a year. 

• Different reapplication policies. The reapplication process varies widely by state. In 13 
states, SES providers never have to formally reapply, and in 12 states, SES providers have 
to reapply every year. 



Introduction 

The No Child Left Behind (NCLB) Act requires Title I schools that have not made ade- 
quate yearly progress (AYP) in raising student achievement for three or more consecutive 
years to offer supplemental educational services (SES) to low-income students. (Title I 
schools are those that receive federal funds to educate disadvantaged children through Title 
I of the Elementary and Secondary Education Act as amended by NCLB.) Supplemental 
services include tutoring, remediation, and other academic interventions designed to 
increase student achievement. These services must be provided outside of the regular school 
day and typically take the form of before- or after-school small group instruction. 

State agencies in charge of education play an important role in identifying and approving 
organizations that are qualified to provide supplemental services, as well as in monitoring 
the quality and effectiveness of the providers. To learn more about the state role in oversee- 
ing supplemental services, the Center on Education Policy (CEP) included several questions 
about these services in our annual survey of NCLB implementation, which we have admin- 
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istered since 2002 to all 50 states.^ The survey, which was conducted in the fall of 2006, 
focused on state implementation of SES in the current school year, 2006-07. Specifically, we 
asked states which criteria they use to review and approve potential SES providers, and how 
often they review SES provider applications, require providers to reapply, and update their 
approved provider lists. We also asked about the extent to which states are able to monitor 
the quality and effectiveness of SES providers. This report describes our survey findings. 



Monitoring the Quality and Effectiveness of SES Providers 

A key question about supplemental educational services is whether providers are helping to 
increase student achievement. Toward this end, NCLB law requires states to monitor the 
quality and effectiveness of approved SES providers in raising achievement. Our survey 
asked states about the extent to which they have been able to carry out this monitoring. 
Only 10 states responded that they are able to monitor the quality and effectiveness of 
providers “to a great extent.” As shown in Figure 1, more than two-thirds of the states said 
they are either somewhat or minimally able to monitor quality and effectiveness, and three 
indicated they are “not at all” able to do this monitoring. 




Figure i. Extent to Which States Are Able to Monitor the Quality and 
Effectiveness of SES Providers’ Services 




Figure reads: Of the 50 state education agencies surveyed by CEP, 19 reported that they have been “somewhat” able to 
monitor the quality and effectiveness of SES providers’ services, 16 said they have been “minimally” able to do this 
monitoring, and 10 said they have been able to do this monitoring “to a great extent.” Three states said they have not 
been able to do this required monitoring at all. 

Source: Center on Education Policy, December 2006, state survey, item 44. 



While all 50 states responded to the overall survey, the response rates for the questions about SES varied from 48 to 49 states. 





To better understand why many states are struggling to monitor the effectiveness of SES 
providers, we asked states to indicate the challenges they are facing or have faced in carrying 
out this monitoring. More than half the states reported that they do not have sufficient staff 
to implement this requirement, and half also indicated that they do not have enough fed- 
eral funding. Table 1 displays these findings and other challenges that states face in moni- 
toring the quality of SES providers. 

Our survey also yielded other interesting findings about state capacity to monitor providers: 

• States varied as to whether insufficient technological capacity was a significant 
impediment to monitoring providers: 22 states viewed it as a moderate or great chal- 
lenge, while 25 said that it was not a challenge or was minimally challenging. 

• Staffing problems may stem from a lack of funding rather than an inability to attract 
and retain staff While most states reported that they did not have sufficient staff to 
monitor providers, only a limited number of states mentioned attracting and retain- 
ing staff as a significant challenge. 

• Most states reported that insufficient guidance from the U.S. Department of 
Education was either a minimal or moderate challenge to monitoring SES providers. 




Table 1. Number of States Reporting Various Factors as Challenges to 
Their Capacity to Monitor SES Providers 



Challenges 1 


Not at all 


Minimally 


Moderately 


To a great 
extent 


Don’t know 


Insufficient numbers of staff to 
implement the requirement 


2 


5 


14 


26 


2 


Inadequate federal funds to 
implement the requirement 


7 


6 


9 


25 


2 


Inadequate state funds to 
implement the requirement 


12 


5 


7 


14 


8 


Insufficient technological capacity 
to implement the requirement 


11 


14 


13 


9 


2 


Insufficient guidance from the 
U.S. Department of Education 


8 


12 


17 


8 


3 


Inability to attract and retain 
qualified staff to implement 
the requirement 


18 


14 


5 


8 


2 



Table reads: Of the 50 state education agencies surveyed by CEP, 26 said that their capacity to monitor the quality and 
effectiveness of SES providers has been affected “to a great extent” by insufficient numbers of staff. Fourteen states said 
that insufficient staff has affected their monitoring moderately and 5 said it has affected their monitoring minimally. 

Source: Center on Education Policy, December 2006, state survey, item 44a. 
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To delve deeper into this issue, we looked at states’ initial responses to the question, To what 
extent is your state able to monitor the quality and effectiveness of approved SES providers? We 
divided the states into two groups: 1) the 19 states that answered “not at all” or “minimally,” 
which we will call states with less capacity, and 2) the 29 states that answered “somewhat” or 
“to a great extent,” which we will call states with more capacity. We then examined how 
these two groups responded to the capacity challenges listed in table 1 . For two of the capac- 
ity challenges — insufficient staff and inadequate state funding — ^we noticed a pattern 
between the two groups. 

States with less capacity reported insufficient staff to be the greatest challenge in monitoring 
SES providers, while states with more capacity tended to see this as a moderate challenge. 
Figure 2 shows the difference between the two groups. 

As shown in Figure 3, states with less capacity reported more often than states with more 
capacity that inadequate state funding challenged their monitoring capacity “to a great 
extent.” States with more capacity reported “not at all” to the same question more often than 
those with less capacity. Both groups, however, identified inadequate federal funding as a sig- 
nificant capacity challenge. 




Figure 2. Extent to Which Insufficient Numbers of Staff Pose a Challenge to 
Monitoring SES Providers 



I I States with less capacity 
^ States with more capacity 




Figure reads: Among the state education agencies with less capacity to monitor SES providers, i6 reported that insuffi- 
cient numbers of staff challenged their monitoring capacity “to a great extent” and only i said it challenged their mon- 
itoring “moderately.” Among the states with more capacity to monitor providers, ii said that insufficient staff 
challenged their monitoring to a great extent and 13 said it challenged their monitoring moderately. 

Source: Center on Education Policy, December 2006, state survey, item 44a. 





Figure 3. Extent to Which Inadequate State Funds Pose a Challenge to 
Monitoring SES Providers 




Figure reads: Among the state education agencies with less capacity to monitor SES providers, 8 reported that inade- 
quate state funds challenged their monitoring capacity “to a great extent” and only 3 said it did not challenge their 
monitoring capacity at all. Among the states with more capacity to monitor providers, 5 said that insufficient staff chal- 
lenged their monitoring to a great extent and 10 said it did not challenge their monitoring at all. 

Source: Center on Education Policy, December 2006, state survey, item 44a. 



State Criteria for Reviewing and Approving Providers 

State education agencies are required by law to develop and apply objective criteria for 
approving potential SES providers. According to non-regulatory guidance issued by the U.S. 
Department of Education,^ states must ensure that each approved provider meets all of the 
following five criteria: 

1 . Has a demonstrated record of effectiveness for increasing student academic achievement 

2. Will use high quality, research-based strategies designed to increase student aca- 
demic achievement 

3. Will provide services that are consistent with school districts’ instructional programs 
and state content and academic achievement standards 

4. Is financially sound 

5 . Will adhere to federal, state, and local health, safety, and civil rights laws 

2 These requirements are also found in the statute but are combined under one section in non-regulatory guidance published on 
June 13, 2005. 
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State education agencies must also ensure that potential SES providers are aware of the appli- 
cation procedures and must publish an updated list of approved SES providers. (Neither the 
law nor the guidance specifies how often the list must be updated.) 

States must make this updated list of approved providers available to parents of eligible students 
who attend Title I schools that are required to offer SES because they have been identified for 
improvement under NCLB for two or more years. (Eligible students are those who qualify for 
free or reduced-price lunch.) Parents choose a provider that will serve their children from the 
state-approved list. Service providers vary considerably and may include public or private (non- 
profit or for-profit) entities such as businesses, national tutoring companies, religious or sectar- 
ian institutions, or community organizations. In 2005, more than half of the SES providers 
nationally were private, for-profit groups, according to data from CEP s annual state survey; 
school districts were the second most common type of provider.^ School districts are allowed to 
provide tutoring services as long as they are not identified for improvement under NCLB. The 
U.S. Department of Education has allowed a few exceptions to this regulation. For example, the 
Chicago school district was allowed to provide SES to its students even though it was identified 
for improvement. 

Our survey asked states to indicate the criteria they use to review and approve potential SES 
providers. Most of the survey responses, listed in Table 2, address the major criteria in federal 
guidance as well as additional criteria states might include. As indicated in table 2, the majority 
of states reported that they do use the NCLB criteria to review and approve providers. 

Only one state indicated that it did not consider whether a provider has a demonstrated record 
of effectiveness in improving student academic achievement when it reviews and approves 
potential SES providers. Likewise, only two states reported not considering whether the provider 
uses research-based strategies as a criterion, even though the NCLB statute and guidance require 
this. No explanations were offered by the states for these responses. 

Eight states said they did not ask providers for evidence that their employees undergo back- 
ground checks, although some of these states explained that the providers assure these checks 
are done and submit documentation to school districts. While background checks are not a spe- 
cific federal requirement, they may be viewed by states as part of the health and safety require- 
ments that SES providers must meet. 

In addition to the required criteria in federal guidance, we asked states if they require potential 
SES providers to indicate during the approval process whether they will serve special popula- 
tions of students, such as students with disabilities or English language learners. Although 
providers are not required to serve these groups, recent studies show that these two groups have 
been difficult to serve through SES.^ Of the 49 states that responded to this item, only 4 
reported they did not use this criterion to review or approve providers. Several states offered 
explanations. For example, one state explained that it encourages organizations that are prepared 
to meet the needs of these students to become providers. 

The recently released report of the Commission on No Child Left Behind, Beyond NCLB, 
included a table about the criteria states used to select SES providers in 2003-04.^ When we 

3 For more detail on types of providers, see Center on Education Policy, From the Capital to the Classroom: Year 4 of the No Child 
Left Behind Act {\Nash\r\gton, DC: CEP, 2006), figure 6-A. 

^ U.S. Government Accountability Office, No Child Left Behind Education Act: Education Actions Needed to Improve Local imple- 
mentation and State Evaluation of Supplemental Education Services (Washington, DC: GAO, 2006). 

^ The report lists the source of the data as the TASSIE state survey and review of online state applications. U.S. Department of 
Education, Office of Planning, Evaluation and Policy Development, Policy and Program Studies Service, Title I Accountability and 
School Improvement from 2001 to 2004 (Washington, DC:20o6). 



Table 2. Number of States Using Various Criteria to Review and Approve 
Potential SES Providers, 2006-07 



Don’t 

Criteria Based on NCLB Law and/or Non-Regulatory Federal Guidance Yes No know 

Provider has a demonstrated record of effectiveness in 48 1 

improving student academic achievement 

Provider will use instructional strategies that are high-quality 49 

Provider will use instructional strategies that are based on research 47 2 

Provider will use instructional strategies that are designed to 49 

increase student achievement 

Provider will provide instructional strategies that are consistent with 48 

the instructional program of the school district and with state academic 
content and achievement standards 

Provider is financially sound 47 1 

Provider will provide supplemental services consistent with the applicable 48 
federal, state, and local health, safety, and civil rights laws 

Provider provides instruction and content that is secular, neutral, 49 

and non-ideological 

Additional State Criteria 



Provider will provide services to special populations of students 45 4 

(e.g., students with disabilities, English language learners) 

Provider has produced evidence of their employees having undergone 37 8 3 

background checks 

Table reads: In reviewing and approving supplemental service providers, 48 of the states surveyed by CEP reported 
that they consider whether the entity seeking approval has a demonstrated record of effectiveness in improving 
student achievement (one of the criteria listed in federal guidance). Forty-nine states also reported considering 
whether the entity provides instruction and content that is secular, neutral, and non-ideological. 



1 

© 



Source: Center on Education Policy, December 2006, state survey, item sp. 
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compared our survey data from the 2006-07 school year with the data in the Commission 
report from 2003-04, we noted an apparent increase over the past three years in the num- 
ber of states using four of the criteria discussed above: (1) a demonstrated record of effec- 
tiveness in improving achievement; (2) secular, neutral, and non-ideological instruction and 
content; (3) services to special student populations; and (4) evidence of employee back- 
ground checks. Table 3 compares the numbers of states using these and other criteria as 
reported by CEP and the NCLB Commission. 



Table 3. Number of States Using Various Approval Criteria as Reported by CEP 
and the NCLB Commission 



NCLB 

Criteria Based on NCLB Law and/or Non-Regulatory CEP Commission 

Federal Guidance 2006-07 2003-04 



Provider has a demonstrated record of effectiveness in 48 29 

improving student academic achievement 




Provider will use instructional strategies that are high-quality* * 49 47 

Provider will use instructional strategies that are based on research* 47 47 

Provider will use instructional strategies that are designed to 49 47 

increase student achievement* 

Provider will provide instructional strategies that are consistent with 48 48 

the instructional program of the school district and with state academic 
content and achievement standards 

Provider is financially sound 47 47 

Provider will provide supplemental services consistent with the applicable 48 46 

federal, state, and local health, safety, and civil rights laws 



Provider provides instruction and content that is secular, neutral, 49 42 

and non-ideological 



Additional State Criteria 



Provider will provide services to special populations of students 45 7 

(e.g., students with disabilities, English language learners) 

Provider has produced evidence of their employees having undergone 37 33 

background checks 



Table reads: Forty-eight of the 50 states surveyed by CEP reported that in school year 2006-07 they considered 
whether an SES provider seeking state approval has a demonstrated record of effectiveness in improving student 
achievement, while 29 states reported considering this criteria in 2003-04 according to the NCLB Commission report. 

* The NCLB Commission report combines these three criteria. 

Note: The shaded rows highlight criteria with notable differences between the CEP data and Commission data. 

Source: Center on Education Policy, December 2006, state survey, item sp; and Commission on No Child Left Behind, Beyond 
NCLB: Fulfilling the Promise to Our Nation’s Children (Washington, DC: 2007). 








There are many possible reasons why CEP s recent data differ from the Commissions ear- 
lier data. Over the years, the numbers of schools and students eligible for SES have 
increased, and one could speculate that states saw the need to become more active in over- 
seeing SES providers. It is also possible that states have become more knowledgeable and 
clear about their authority and responsibilities to oversee providers — especially since the 
publication of federal guidance on SES in 2005 — and have become more sophisticated in 
their capacity to evaluate providers based on certain criteria. 



Frequency of Application Reviews, Reapplication, and Updates 

The NCLB law requires state education agencies to consult with parents, teachers, school 
districts, and interested members of the public to promote maximum participation by 
providers and give parents as many choices as possible. Furthermore, states must maintain 
an updated list of approved SES providers from which parents may select. Therefore, we 
asked states how often they review new SES provider applications, require approved 
providers to reapply, and update the approved SES provider lists. 

Figure 4 shows the number of states giving various responses to the question of how often 
they review new provider applications. Twenty-nine states said they reviewed new applica- 
tions once a year, 1 1 said twice a year, and 6 reported reviewing applications on a rolling 
basis (as they are received). 

State responses also varied as to how often they require SES providers to reapply once they 
are approved. As shown in Figure 5, 13 states never require providers to reapply, 12 states 
require reapplication annually, and 9 require reapplication every two years. It is important 
to note that some states may require SES providers to submit additional documentation 
annually even though they may not have to formally reapply. 

The remaining states have other reapplication cycles. For example, one state requires condition- 
ally approved providers to reapply every year until full approval is granted, but fully approved 
providers only need to reapply every four years. Further, some states rely on other factors to 
determine which providers need to reapply, such as how well providers are doing according to 
evaluations of effectiveness and whether providers comply with applicable requirements, provide 
continuous services, or make changes in their programs. One state has not established a reappli- 
cation schedule, and another offered no explanation for responding “other.” 

It is also important to know how often states update their SES provider list because school 
districts and parents rely on state education agencies for the most current information about 
providers that are approved to serve students. In response to our survey question, 28 states 
reported that they update their provider list annually. In addition, 22 states update their lists 
more frequently. Eight update their lists twice a year, while two do it every quarter. Nine 
states update their list on a rolling basis (in other words, as soon as new providers are 
approved), and three do it as needed to keep up with changes, such as the removal of a 
provider or a change in a provider s contact information.^ 




^ These three states include one of the 28 states that updates its list annually as well. 
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Figure 4. How Often Are New SES Provider Applications Reviewed by States? 




Number of States 



11 




^ Once a year 
^ Twice a year 
Quarterly 
I I Rolling basis 

I I Other 



Figure reads: Twenty-nine of the 50 state education agencies surveyed by CEP reported that they review new applica- 
tions from potential SES providers once a year. 

Source: Center on Education Policy, December 2006, state survey, item 40. 



Figure 5. How Often Do Approved SES Providers Have to Reapply? 




Never Annually Every 2 Every 3 Other 

years years 



Figure reads: Thirteen of the 50 state education agencies surveyed by CEP reported that they never require SES providers 
to reapply for state approval. 



Source: Center on Education Policy, December 2006, state survey, item 40. 







Conclusion 



Monitoring the quality and effectiveness of the tutoring services offered by SES providers is 
a capacity challenge for many states. Most state education agencies are only somewhat or 
minimally able to carry out this monitoring, according to our survey. The biggest challenges 
states face in monitoring SES providers appear to be an insufficient number of staff and 
inadequate federal funding. 

Overall, the vast majority of state education agencies reported that they are using the crite- 
ria in the NCLB non-regulatory guidance to review and approve potential SES providers. 
Most states review new applications frequently, either once a year or more often, and the 
majority of states reported updating their approved SES provider lists once a year. However, 
the frequency with which SES providers had to reapply varied across states. 

While this report presents information about how states are implementing SES, further 
research is needed in order to better understand the challenges states face. Additional infor- 
mation about SES program implementation and effectiveness could shed light on whether 
supplemental educational services are indeed contributing to higher academic achievement 
for disadvantaged children. 



Research Methods 

This report is part of a larger study examining state capacity in implementing NCLB. 
Our study focuses on two main research questions: In what ways are states fulfilling their 
responsibilities under NCLB, and to what extent do state education agencies have the 
capacity to implement the requirements of NCLB? 

In August 2006, the Center on Education Policy staff contacted the chief state school 
officers in each of the 50 states and the District of Columbia requesting their participa- 
tion in a survey on NCLB to be administered in fall 2006. The District of Columbia did 
not respond to our request to participate. We also asked each chief to designate an indi- 
vidual within the state education agency as the primary contact for the survey. In addi- 
tion, we asked the Georgia Department of Education to pilot the survey instrument, 
which it did in September 2006. In October 2006, CEP staff sent the revised survey 
instrument, containing 64 questions, to state contacts by e-mail. States returned the sur- 
veys to CEP from October 2006 through January 2007. All 50 states returned a com- 
pleted survey although some states did not complete every question or section. Thus our 
response rate varies by question. 
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